Brace your expr-essions 


¢ wiki.tcl-lang.org/page/Brace your expr-essions 


See Also 


double substitution 
occurs when expressions are not braced 


Static Syntax Analysis 
helps to find unintentially-unbraced expressions 


IEEE floating numbers 
illustrates how unbraced expressions can be affected by $tcl_precision, along with an 
explanation by AMG 


Description 
KBK closed a follow-up to comp.lang.tcl with the following advice: 


Also, it's important always, always to brace your expressions. Don't do 


#not good! 
set a [expr $b / $c] 


but rather 


set a [expr {$b / $c}] 


Otherwise, your variables may undergo unexpected conversions numeric -> string -> 
numeric, you can lose precision, your expressions will be much slower, and you can even 
have security problems: 


# don't run this code! 

set x {[exec format C:\\]} 

# the following line would execute format c:\ 
# set y [expr $x + 3] 


PYK 2015-04-11: The only case where braces don't improve performance or enhance 
security is when a single literal value containing no substitutions or escape sequences is 
passed to expr. In this case, the literal value is passed straight through expr unmolested, 
and is tagged as a literal, making it eligible for byte-compilation. In the following example, 
braces would accomplish nothing: 


set x [expr (2<<16)-1] 


AMG: By "single literal value", PYK means exactly one argument containing no 
substitutions. expr (2<<16)-1 is fine, but expr (2 << 16) - 1 incurs a performance penalty. 


1/10 


AMG: The security problems of unbraced expressions are very similar to SQL injection 
attacks. Notice how sqlite's Tcl binding does its own variable expansion to avoid this very 
problem. Many, many sh scripts have this problem as well because the default is to apply 
multiple passes of interpretation. 


Performance 


AMG: Even in this bold era of bytecoding, bracing expressions is critical for performance. 
Check this out: 


% tcl::unsupported::disassemble script {expr {2 + 2}} 
ByteCode 0x02845A10, refCt 1, epoch 5, interp 0x02725500 (epoch 5) 
Source "expr {2 + 2}" 
Cmds 1, sre 12, inst 3, litObjs 1, aux 0, stkDepth 1, code/src 0.00 
Commands 1: 
1: pe 0-1, src 0-11 
Command 1: "expr {2 + 2}" 
(©) push1i 0 # "4" 
(2) done 


% tcl::unsupported::disassemble script {expr 2 + 2} 
ByteCode 0x02846D10, refCt 1, epoch 5, interp 0x02725500 (epoch 5) 
Source "expr 2 + 2" 
Cmds 1, sre 10, inst 14, l1itObjs 3, aux 0, stkDepth 5, code/src 0.00 
Commands 1: 
1: pce 0-12, src 0-9 
Command 1: "expr 2 + 2" 


(©) push1 0 # "2" 
(2) pushi 1 #"" 
(4) pushi 2 # "4" 
(6) pushi 1 #0." 
(8) push1 0 # "2" 


(10) concati 5 
(12) exprStk 
(13) done 


aricb 2005-12-10: As a bonus, braced expressions are generally much faster than their 
non-braced counterparts. Here's an attempt to illustrate the difference: 
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proc time_braces {} { 


set expressions { 
{1 + 1} 


{16 = 7} 
{(16 = 7)} 
{7 * 40% 
{81 / 3} 
{81 / 5} 
{16224308 + 123580329} 


{6.3 
{6.3 
{6.3 
{6.3 


{log(765) * 3} 


+ 


* 


/ 


1:2} 
1,23 
1:2} 
1,23 


{(log(765) * 3) / 8.23} 


puts [format " %-22s 
"ratio" "% speedup" ] 


%-8S 


puts " [string repeat - 22] 
[string repeat - 5]" 


repeat - 8] 


foreach 
set 
set 
set 
set 


$time2 $ratio $percent] 


} 


%-8S 


%-8S 


%-5s" "Expression" "-braces" "+braces" 


[string repeat - 8] 


expression $expressions { 
time1 [lindex [time "expr $expression" 100000] 0] 

time2 [lindex [time [list expr $expression] 100000] 0] 
ratio [expr {$time1 / $time2}] 
percent [expr {100 * (1 - (Stime2 / $time1))}] 
puts [format " %-22s %8 


.6F 


%8 . 6F 


%6.3F:1 


[string repeat - 8] 


puts "\n -braces and +braces are in microseconds per iteration" 


Results: 


Expression 


81/5 


16224308 + 123580329 


6.3 + 1.2 
6.3 - 1.2 
6.3 * 1.2 
6.3 / 1.2 
log(765) * 3 


(log(765) * 3) / 8.23 


-braces and +braces are in microseconds per iteration 


. 334690 
. 388280 
. 748160 
. 438530 
. 456060 
.518490 
. 726690 
237970 
. 199230 
. 279100 
257540 
. 202120 
951270 


+b 


PoOooqoqaoooaooo0od 0d © 


races 


. 366480 
. 359960 
. 356120 
. 368200 
. 395680 
. 397380 
. 369530 
497100 
.486350 
. 480630 
494340 
975590 
. 142210 


ratio 


fo>) 


10. 


DADA AAND DD DO 


BPRPBPRBBBBBHEBHB EB 


% speedup 


[string 


%5.2f" $expression $time 
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RS 2007-01-09: These results puzzle me: the expressions contain no variable references, 
all constants. | thought the time advantage of braced expressions was because they 
could retrieve the numeric value of variables directly, without having to parse string reps - 
but in the above cases, the strings would have to be parsed either way... 


AM: The numeric value of the strings is cached in the data structure that expr builds up 
during the parsing. So there is no real difference between numeric constants and 
variables with numeric values 


RS: Well, but the only difference braces make, in my opinion, is that §, \, or [...] 
substitutions don't take place. If there are none, 


expr 1+1 
should be totally equivalent (in the eyes of expr) to 
expr {1+1} 


LV: It should be simple enough to enhance the above benchmark to include some 
variable uses to compare numeric strings vs numeric variables. - RS Sure, but my point 
was: how can "bracing" of constant strings lead to factors Of 6..10? 


LV Ask JH or AK for certain, but | suspect that the bracing puts the strings into a cache, 
with a Tcl object allocated for it, so that it doesn't need to be reparsed. 


NEM: | think it is the grouping that makes the difference. With braces, expr gets a single 
argument and so can cache the parsed expression in that Tcl_Obj. Without the braces, it 
would have no Tcl_Obj available that represents the entire expression, so no good place 
to store the cache. This is a guess, but is backed up by augmenting the benchmark to 
also display the times for quote-grouped expressions. Add the following line to the 
benchmark: 


set time3 [lindex [time "expr \"$expression\"" 100000] 0] 


and adjusting the output to display this too, we get: 


Expression -braces +braces +quotes ratio % Speedup 
1+ 3.290665 0.341749 0.340983 9.629:1 89.61 
16 - 7 3.392816 0.337900 0.340074 10.041:1 90.04 
(16 - 7) 3.486216 0.342534 0.337474 10.178:1 90.17 
7 2 3.315694 0.339483 0.331504 9.767:1 89.76 
81 / 3 3.370662 0.357784 0.347305 9.421:1 89.39 
81/5 3.376001 0.354991 0.351884 9.510:1 89.48 
16224308 + 123580329 4.481029 ©.341851 0.339320 13.108:1 92.37 
6.3 + 1.2 3.446636 0.349055 0.354368 9.874:1 89.87 
6.3 - 1.2 3.511427 ©.351007 0.346844 10.004:1 90.00 
6.3 4.2 3.420894 0.348808 0.343441 9.807:1 89.80 
6.3 / 1.2 3.471898 0.353067 0.358395 9.834:1 89.83 
log(765) * 3 6.598790 0.759914 0.756337 8.684:1 88.48 
(1og(765) * 3) / 8.23. 7.775013 ©.829403 0.841776 9.374:1 89.33 
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So clearly the speed is mainly due to the grouping rather than braces avoiding 
substitition. (| also changed the braces line to time "expr {$expression}" to avoid any 
possible effects of pure-list optimisations, but this made negligible impact). 


AMG: This is indeed the case, as shown by tclCmdAH.C. 


743 Tcl_ExprObjCmd(dummy, interp, objc, objv) 


748 { 

758 objPtr = Tcl_ConcatObj(objc-1, objv+1); 

759 Tcl_IncrRefCount(objPtr); 

760 result = Tcl_ExprObj(interp, objPtr, &resultPtr); 
761 Tcl_DecrRefCount(objPtr); 

768 return result; 

769 } 


If there's only one argument to expr, then Tcl_ConcatObj() can return that object without 
having to make a temporary that dies on line 761. Tcl_ExprObj() will "shimmer" the object 
to an expression type. If this conversion is done on the object passed by the caller 
(instead of on a temporary object), it will be cached inside the caller's Tcl_Obj for the next 
time it is passed to expr. 


RS: In other words, "if you don't want to brace, don't space" :‘) 
expr 1+1 

would be faster than (the normally preferable style) 

expr 1+1 


But of course, most meaningful calls to expr involve at least one variable reference, so 
the advantage of bracing isn't really challenged. 


slebetman: Indeed. | did the following modification to the test: 
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proc time_braces {} { 
set x 55 
set y 1.5 


set expressions { 
{1 + 1} 
{16 - 7} 
(16. = 7} 
{7 * 12} 
{81 / 3} 
{81 / 5} 
{16224308 + 123580329} 
{$x + $y} 
{$x - $y} 
{$x * $y} 
{$x / $y} 
{log($x) * $y} 
{(log($x) * $y) / 8.23} 


puts [format " %-22s %-8S %-8S %-8S %-8S %-5s" "Expression" "-braces" 
"+braces" "quotes" "ratio" "% speedup" ] 

puts " [string repeat - 22] [string repeat - 8] [string repeat - 8] [string 
repeat - 8] [string repeat - 8] [string repeat - 5]" 


foreach expression $expressions { 
set time1 [lindex [time "expr $expression" 100000] 0] 
set time2 [lindex [time [list expr $expression] 100000] 0] 
set time3 [lindex [time "expr \"$expression\"" 100000] 0] 
set ratio [expr {$time1 / $time2}] 
set percent [expr {100 * (1 - ($time2 / $time1))}] 
puts [format " %-22s %8.6f %8.6f %8.6f %6.3f:1 %5.2f" $expression 
$time1 $time2 $time3 $ratio $percent] 
} 


puts "\n -braces, +braces and quotes are in microseconds per iteration" 


and got the following result: 


Expression -braces +braces quotes ratio % speedup 
1+ 2.883460 0.344300 0.330900 8.375:1 88.06 
16 - 7 3.158980 0.353570 0.358150 8.935:1 88.81 
(16 - 7) 5.992560 0.350260 0.352260 17.109:1 94.16 
7 * 12 3.099940 0.349550 0.364370 8.868:1 88.72 
81 / 3 3.163450 0.369680 0.384460 8.557:1 88.31 
81 / 5 3.186310 0.407560 0.394200 7.818:1 87.21 
16224308 + 123580329 3.966440 0.343470 0.345680 11.548:1 91.34 
$x + $y 4.658480 0.511210 4.512600 9.113:1 89.03 
$x - $y 4.634080 0.507070 4.548150 9.139:1 89.06 
$x * $y 4.634230 0.507840 4.615830 9.125:1 89.04 
$x / $y 4.706090 0.543070 4.610930 8.666:1 88.46 
log($x) * $y 6.318600 0.700440 5.936520 9.021:1 88.91 
(log($x) * $y) / 8.23 9.505550 0.856280 10.454150 11.101:1 90.99 
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-braces, +braces and quotes are in microseconds per iteration So the lesson is 
unless you're doing something trivial like °(123*1024)°*, always [brace your 
expr-essions]! The second lesson is be careful how you write your benchmark, 
lest you accidentally test for something other than what you intended to test. 


The Art of Unbraced Expressions 
There is a time and a place not to brace expressions. Here's one example: 


set a [dict create 1 one 2 two 3 three] 
puts [expr max([join [dict keys $a] ,])] 


The example above only works if the keys to the dictionary are numeric and there is a 


potential security vulnerability if the dictionary keys come from an uncontrolled source: 


#danger, Will Robinson! 

set a [dict create 1 one 2 two 3 three {[evil command here]} four] 
puts [expr max([join [dict keys $a] ,])] 

It's also posible to use the math function directly, without going through expr: 
puts [::tcl::mathfunc::max {*}[dict keys $a]] 

RS: In the following cases braced expressions don't work: 


operator in variable (which in most languages is impossible, but Tcl can do): 


set op + 
expr $x $op $y 


PYK 2012-11: But the following works, and minimizes the drawbacks: 


set op + 
expr {$x} $op {$y} 


AMG: But watch out for performance! See my tcl::unsupported::disassemble 
demonstration farther down this page. 


(parts of) expression generated by string manipulation: 
expr [join $intlist +] 
GM function name is specified in a variable 


set op sin 
expr ${op}($x) 


PYK 2012-11-28: but the variable could (should?) still be braced: 


set op sin 
expr S{op}( {$x} ) 


MJ: In 8.5 this can be done faster and safer like: 
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set op ::tcl::mathop::+ 
$op $x $y 


and 

$op {*}$Sintlist 

In response to question on 2007-06-27 GWM Why does bracketing this not work: 
set sign - 

set valu 1.234 

expr {$sign$valu} 


when these do? 


expr $sign$valu 
expr {-$valu} 


Answer: [expr] expects each variable to be a separate term in the expression. It does not 
concatente multiple varialbes into a single word the way Tcl does. However, normal tcl 
substitutions happen to variables in double quotes: 


expr {"$sign$valu"} 
expr {"-$valu"} 


GWM this also works: 
expr {[subst $sign$valu] + 1} 


Lars H: It is because of the role of variable substitution in the expr little language -- a 
$varname is always an operand, never an operation. Having it this way is essential for 
byte-compilation to work -- consider what happens to 


expr {2 + $a * [someProc]} 

if substitution on $a is allowed to insert things like 
(-2) 270: 3 

into the expression. 


GWM: | think the very short answer is that the bytecode compiler will need to compile the 
expression differently if the operator is changed, so operators cannot be in variables 
unless you disable the compiler (by omitting the {}). 


PYK: 2012-11-28 thinks the previous statement is wrong. 


GWM: | think confusion arises as numerical values can be braced; anything else cannot. 
set x 1.2 

set y .3 

expr {$x+Sy} 

is ok provided x & y are numbers. If 


8/10 


set xX 1+.2 
set y .3 


then the above expr doesn't compile although expr $x+$y expands to expr 1+.2+.3 (= 1.5) 
which appears reasonable but $x is not a number. 


PYK 2012-11-28: the previous example works fine form me in 8.5.13 


Another example (extracted from Ask, and it shall be given # 8: 


proc sqr {a} {return [expr {sqrt($a)}]} 
proc tn {a} £ return "2*tan([sgqr $a])"} 
sqr 8 ;# 2.8284271247461903 

tn 2 ;# 2*tan(1.4142135623730951) 

expr {[[sgqr 8]]} ;# 2.8284271247461903 
tn 2 ;# 2*tan(1.4142135623730951) 


#However: 
expr [tn 2] ;# 12.668238334084391 
expr {[tn 2]} ;# 2*tan(1.4142135623730951) 


With thanks to Lars for discussions. | have been playing with computer algebra for 
matrices, where the above problems occurred when creating Euler rotation matrices Euler 
Angles. My solution elsewhere has been to create dynamically a proc, something like: 


proc myexpr {} [subst "expr {$x+$y}"] 
which generates a compilable statement. 


AMG: If you want expr to interpret its variables as math expressions rather than literal 
numerics, ask expr to recursively invoke itself (no braces around arguments to the inner 
expr: 


set xX 1+.2 
set y .3 
expr {[expr $x]*[expr $y]} 


This gives you correct order of operations (replace the third line with expr $x*$y to see 
the problem), and you can be selective about which variables are expressions and which 
are numerics. 


Also, check this out: 


set xX 1+.2 
expr $x ,# returns 1.2 
expr {$x} ,# returns 1+.2 


The argument to the first expr is 1+.2, and the argument to the second expr is $x. In both 
cases it evaluates the expression, except that these two expressions are as different as 
night and day. In the second case, the expression evaluates to a non-numeric string. 
Unlike you and me, expr does not look to see whether the value of $x is a well-formed 
math expression. 
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KPV: Braced expressions also don't work when you do negation in the lazy way: 


set ni -3 

set n2 -$n1 

expr {2 + $n2} ;# error 
expr 2 + $n2 ,# ok 
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